Papers with Jensen-Shannon divergence
TokCollate: A Comprehensive Tool for Tokenizer Evaluation and Visualization across Languages (2026.acl-demo)
Copied to clipboard
| Challenge: | Tokenization quality varies significantly across languages, leading to disparities in LLM performance and cost for speakers of less-resourced languages. |
| Approach: | They propose a Python-based evaluation framework and a JavaScript visualization interface that evaluates tokenizers in a variety of languages. |
| Outcome: | TokCollate is an evaluation framework for tokenizers with a JavaScript visualization interface. |
DivMerge: A divergence-based model merging method for multi-tasking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for multitask learning struggle with interference between tasks, especially as the number of tasks grows. |
| Approach: | They propose a reference-free method that minimizes the divergence between models' outputs and those of the merged model, automatically balancing task importance. |
| Outcome: | The proposed method outperforms existing methods on classification and generative tasks and remains robust when scaling to more tasks. |
Sparse Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text generators require sampling from a modified softmax to avoid degenerate text . entmax sampling creates a mismatch between training and testing conditions . |
| Approach: | They propose to use entmax transformation to train and sample from a sparse language model to avoid degenerate text. |
| Outcome: | The proposed model improves fluency and consistency, fewer repetitions, and n-gram diversity closer to human text. |
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing contrastive methods that ignore the context of a large language model (LLM) fail to handle instances that vary in their amount of conflict, with static methods over-adjusting when conflict is absent. |
| Approach: | They propose a fine-grained, instance-level approach called AdaCAD which dynamically adjusts the degree of conflict based on the degree. |
| Outcome: | The proposed approach outperforms baselines and improves factuality of summaries by 6.19. |
Diverging Divergences: Examining Variants of Jensen Shannon Divergence for Corpus Comparison Tasks (2020.lrec-1)
Copied to clipboard
| Challenge: | Jensen-Shannon divergence (JSD) is a distribution similarity measurement widely used in natural language processing. |
| Approach: | They propose to use a weighted version of Jensen-Shannon divergence to compare corpora . they argue this weighting is unnecessary and can lead to misleading results . |
| Outcome: | The proposed weighting is unnecessary and can lead to misleading results. |
Distributional Alignment for Large Language Models under Domain Shift (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing distributional alignment models are unstable and degrade under cultural and domain shifts. |
| Approach: | They propose a distributional alignment technique that improves distribution prediction under cultural and domain shift. |
| Outcome: | The proposed method improves fidelity and robustness of LLM distribution estimation under domain and cultural shift. |
Constrained Non-negative Matrix Factorization for Guided Topic Modeling of Minority Topics (2025.emnlp-main)
Copied to clipboard
| Challenge: | Topic models often fail to capture low-prevalence, domain-critical themes such as mental health themes in online comments. |
| Approach: | They propose a topic modeling solution using a constrained NMF to capture minority topics . they incorporate a seed word list characterizing minority content of interest . |
| Outcome: | The proposed model outperforms baselines on synthetic data in terms of topic purity and quality using Jensen-Shannon divergence. |